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ABSTRACT 

Analysis of strong gravitational lensing data is important in this era of precision cosmology. The 
objective of the present study is to directly compare the analysis of strong gravitational lens systems 
using different lens model software and similarly parameterized models to understand the differences 
and limitations of the resulting models. The software lens model translation tool, HyclraLens, was used 
to generate multiple models for four strong lens systems including COSMOS J095930+023427, SDSS 
J1320+1644, SDSSJ1430+4105 and J1000+0021. All four lens systems were modeled with PixeLens, 
Lenstool, glafic, and Lensmodel. The input data and parameterization of each lens model was similar 
for the four model programs used to highlight differences in the output results. The calculation of 
the Einstein radius and enclosed mass for each lens model was comparable. The results were more 
dissimilar if the masses of more than one lens potential were free-parameters. The image tracing 
algorithms of the software are different, resulting in different output image positions and differences 
in time delay and magnification calculations, as well as ellipticity and position angle of the resulting 
lens model. In a comparison of different software versions using identical model input files, results 
differed significantly when using two versions of the same software. These results further support the 
need for future lensing studies to include multiple lens models, use of open software, availability of 
lens model files use in studies, and computer challenges to develop new approaches. Future studies 
need a standard nomenclature and specification of the software used to allow improved interpretation, 
reproducibility and transparency of results. 

Subject headings: strong gravitational lens models, direct comparison studies, indirect comparison 
studies, lens model software 

1. INTRODUCTION 

The present time has been referred to as the ’’Golden 
Age” of Precision Cosmology. Strong gravitational lens¬ 
ing data is a rich source of information about the struc¬ 
ture and dynamics of the universe, and these data are 
contributing significantly to this notion of precision cos¬ 
mology. Strong gravitational lens studies are highly de¬ 
pendent on the software used to create the models and 
analyze the components such as lens mass, Einstein ra¬ 
dius, time delays etc. A comprehensive review of avail¬ 
able software has been conducted by Lefor et al. (2013). 

While many such software packages are available, most 
studies utilize only a single software package for analy¬ 
sis. Furthermore, most authors of strong gravitational 
lensing studies use their own software only. 

More recently, the status of comparison studies of 
strong gravitational lens models has been reviewed by 
Lefor and Futamase (2013). This study demonstrated 
that changes in redshift affect time delay and mass cal¬ 
culations in a model dependent fashion, with variable 
results with small changes in redshift for the same mod¬ 
els. 

An important resource for the conduct of comparison 
studies is the Orphan Lens Project, a compendium of 
information about strong lens systems that as of May 
2014 contained data for 656 lens systems (Moustakas and 
Brownstein 2013). There are a number of barriers to the 
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conduct of lens model comparisons. Ideally, a compari¬ 
son study of a previously studied lens would include the 
original model for comparison, but this is sometimes im¬ 
possible because the lens model code is not made publicly 
available. Another barrier to performing comparative 
studies is the complexity of the lens model files, since 
there are major differences among the commonly used 
model software available. In order to facilitate this step 
of the process the HydraLens program was developed to 
generate model files for multiple strong gravitational lens 
model packages (Lefor 2014). 

To date, the largest comparison study of strong grav¬ 
itational lens models was an analysis of MACSJ1206.2- 
0847 as part of the CLASH survey conducted by Umetsu 
et al. (2012). This study included four different strong 
gravitational lens models including Lenstool ((Julio et al. 
2007)), PixeLens ((Saha et al. 2006)), LensPerfect (Coe 
et al. (2008)) and SaWLens (Merten et al. (2009)). The 
authors conducted five lens model analyses using the 
same data, and is thus categorized as a direct and semi¬ 
independent study. This type of study has great advan¬ 
tages in that all data and all models are available for 
direct comparison in a single study. 

The Hubble Space Telescope (HST) Frontier Fields 
project is reporting preliminary results 2 (Laporte et al. 
2014). This important deep field observing program com¬ 
bines the power of the HST with gravitational lenses. 

2 http://www.stsci.edu/hst/campaigns/frontier-fields/ 
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Lensing analysis in the Frontier Fields project includes 
models from a number of software codes including ZB, 
GRALE, Lenstool, and two other non-LTM lens model 
software codes which facilitate direct comparison of re¬ 
sults from a number of lens models rather than depending 
on a single model from which to draw conclusions. The 
Hubble Frontier Fields analysis uses models that are in¬ 
dependently developed and optimized by each group of 
investigators for each code used. The power of this ap¬ 
proach has been reported, with more results surely to 
follow (Coe et al. 2015). 

The goal of this study is to directly compare the re¬ 
sults of calculations among four model software codes in 
the evaluation of four lens systems. The present study 
has several unique features. This study is the first to 
use computer-aided lens model design, using HydraLens 
software to facilitate lens model generation.There are no 
previous single studies which compare the results for mul¬ 
tiple lens systems using multiple lens model software. 
This study was designed to further evaluate comparative 
lens model analyses and includes both direct and indirect 
semi-independent studies of four lens systems using four 
different software models. Other studies have included 
indirect comparisons to previous lens model analyses, or 
direct comparisons of several lens models of a single lens 
system. This is the first study to also include combined 
indirect and direct analyses where previously published 
lens models were used for direct comparisons. 

The nomenclature of lens model comparison studies, 
lens systems studied, previous lens model studies of these 
systems and the lens model software used are described 
in section §2. The results of the lens model studies for 
each of the four systems studied are presented in section 
§3 and a review of existing comparison studies along with 
the results of this study are presented in §4. Conclusions 
and suggestions for future lens model studies are in sec¬ 
tion §5. 

2. METHODS 

2 . 1 . Nomenclature 

The use of standardized nomenclature to describe lens¬ 
ing studies is useful to evaluate multiple studies. In 
this paper we follow the nomenclature previously de¬ 
scribed (Lefor and Futamase 2013). Lens model com¬ 
parison studies are referred to as direct when the com¬ 
parison is made based on calculations using two soft¬ 
ware models in the same paper, and indirect when com¬ 
parison is made to previously published data. In this 
study, we also use the actual models from published stud¬ 
ies (kindly supplied by the investigators) so these are 
considered combined indirect/direct comparisons. Lens 
model comparisons using the same data are referred to as 
semi-independent, and when different data is used, the 
comparison is independent. Lastly, software is classified 
as Light Traces Mass (LTM, formerly known as para¬ 
metric), or non Light Traces Mass (non-LTM, formerly 
known as non-parametric). 

2 . 2 . Lens Model Preparation 

Each lens model software package uses a different input 
data format to describe the lens model. All of them 
use simple text files as input, but the format of the text 
files, available functionality and command structures are 


dependent on the particular software. Some lens model 
software uses multiple accessory files to provide other 
data. Each of them has a unique list of commands, with 
great variability. HydraLens (Lefor 2014) 3 was written 
to simplify the process of creating lens model input files 
to facilitate direct comparison studies, and to assist those 
starting in the field. 

The four lens systems were evaluated using four 
lens model codes, necessitating 16 different models. 
The Lenstool model for COSMOS J095930+023427 was 
kindly provided by Cao (Cao et al. 2013). The glafic 
model for SDSS J1320+1644 was kindly provided by 
Rusu (Rusu et al. 2013). The remaining 14 models were 
written for this study using HydraLens. In the case of 
COSMOS J095930+023427 and SDSS J1320+1644, the 
two lens models we received from other investigators were 
used as input to HydraLens which generated the models 
for the other three software packages used in this study. 
In the case of SDSS J1430+4105 and J1000+0021, mod¬ 
els were first written for PixelLens 4 . HydraLens was 
then used to translate the PixeLens model into the for¬ 
mat for the other strong gravitational lens model soft¬ 
ware, including Lenstool , Lensmodel 6 and glafic 1 . 
The translated files output from HydraLens were edited 
to assure that parameters were fixed or free as appro¬ 
priate, and that optimization parameters were correctly 
set. The lens model files were then used as input to the 
respective lens model software. 

2.3. Gravitational Lenses Studied 

The parameters used for the four lens systems was 
obtained from previous studies. The geometry for 
each system was identical in all four models evalu¬ 
ated, and therefore all studies conducted are classi¬ 
fied as semi-independent lens analyses. Three of the 
lens systems studied are listed in the Orphan Lens 
Database (Moustakas and Brownstein 2013) includ¬ 
ing COSMOS J095930+023427, SDSS J1320+1644 and 
SDSS J1430+4105. 

2.3.1. COSMOS J095930+023427 

The lens COSMOS J095930 was first described by 
Jackson (Jackson 2008). COSMOS J095930 is an early- 
type galaxy with four bright images of a distant back¬ 
ground source. It is located at zi ens =0.892, and the 
background source is estimated at z source = 2.00. While 
the exact z source is unknown, the value used by previous 
investigators is 2 . 00 . 

Models of this system were described by Faure using 
Lenstool (Faure et al. 2011). This model used a Singular 
Isothermal Ellipsoid (SIE) with external shear (+ 7 ) and 
found an Einstein radius of 0.79” and oy=255 km s _1 . 

More recently, an extensive multi-wavelength study of 
this system was reported by Cao and colleagues (Cao 
et al. 2013), also using Lenstool. This analysis used four 
different models, an SIE with two Singular Isothermal 
Spheres (SIS) as well as a Pseudo-Isothermal Elliptical 
Mass Distribution (PIEMD) model with two SIS, both 

3 http://ascl.net/1402.023 

4 http://ascl.net/1102.00T 

5 http://ascl.net/1102.004 

6 http://ascl.net/1102.003 

7 http://ascl.net/1010.012 
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with and without external shear (Kassiola and Kovner 
1993). We selected the SIE+SIS+SIS model used by Cao 
as the basis of the present indirect comparison with their 
work as well as the direct comparisons with the four lens 
models studied here. 

The Lenstool model developed by Cao and coworkers 
was kindly supplied for this study and used as a baseline 
model which was then translated into input files for the 
other software by HydraLens. The Lenstool model used 
by Cao included priors for the values of ellipticity (e = 
[0.0, 0.9]) and position angle (PA= [-90,90] for the SIE 
potential) and for the velocity dispersion (er = [100,1000] 
for all three potentials). These same priors were used 
in the models of COSMOS J095930 for Lensmodel and 
glafic in this study. The Lenstool model developed by 
Cao uses optimization in the source plane. The image 
positions used in all models of this system were taken 
from Table 1 in Cao (Cao et al. 2013) . 

The Lenstool model developed by Cao has five free 
parameters including the velocity dispersion of the three 
galaxies, and orientation and ellipticity of the SIE galaxy. 
The positions of the second and third galaxies (SIS) in 
the model were fixed. The models used here were simi¬ 
larly parameterized. 

The present study is an indirect comparison with the 
analysis of Cao (Cao et al. 2013) as well as a direct 
comparison of the four lens models studied. Since we 
were provided the model used by Cao, it is a com¬ 
bined indirect/direct comparative analysis of COSMOS 
J095930. All four models of this system used a D m = 0.3, 
Da = 0.7, H 0 = 70 km s _1 cosmology, as was used by 
Cao et al. (2013). 

2.3.2. SDSS J1320+16U 

SDSSJ1320+1644 was initially described by Oguri 
et al. (2012) and Inada et al. (2012), and is a large separa¬ 
tion lensed quasar candidate identified in the SDSS, with 
a separation of 8".585 ±0".002 at z source = 1.487 (Rusu 
et al. 2013). Both an elliptical and disk-like galaxy were 
identified almost symmetrically between the quasars at 
redshift Zz erls =0.899. 

A detailed lens model analysis of this system was con¬ 
ducted by Rusu et al. (2013), using glafic software. Based 
on their analysis, they conclude that SDSSJ1320+1644 is 
a probable gravitationally lensed quasar, and if it is, this 
would be the largest separation two-imaged lensed quasar 
known. They show that the gravitational lens hypothesis 
implies that the galaxies are not isolated, but are embed¬ 
ded in a dark matter halo, using an NFW model and an 
SIS model. The SIS model has a ov=645±25 km s -1 . 
We use the ’SIS free’ model as the basis of the compari¬ 
son study, as defined by Rusu et al. (2013), which models 
the three galaxies (referred to as Gl, G2 and G4) as SIS 
potentials and leaves the position of the dark matter halo 
(also modeled as a SIS) as a free parameter. The model 
used by Rusu includes priors for the velocity dispersion 
of the dark matter halo (a = [400,800]). The same pri¬ 
ors were used in the models of SDSS J1320+1644 in this 
study. The analysis by Rusu uses optimization by glafic 
in the image plane. The image positions of the two im¬ 
ages were used directly as described by Rusu (Rusu et al. 
2013) . 

Rusu considers models with 0 degrees of freedom, in¬ 
cluding 14 nominal constraints and the same number of 


nominal parameters, which fit with \ 2 << 1- The ellip¬ 
ticity and position angle are used when the position of the 
dark matter halo is fixed. The models developed for this 
study were similarly parameterized using the position of 
the dark matter halo as a free parameter (” SIS-free”) and 
fixed to introduce ellipticity and position angle. 

A number of glafic models developed by Rusu and 
coworkers were kindly supplied for this comparative anal¬ 
ysis and used as a baseline model which was then trans¬ 
lated by HydraLens into models for the other software. 
The present study includes an indirect comparison with 
the analysis of Rusu (Rusu et al. 2013) as well as a di¬ 
rect comparison of the four software lens models stud¬ 
ied. Since we were provided a model used by Rusu, 
this is a combined indirect/direct comparative analysis 
of SDSSJ1320+1644. All four models of this system used 
a D m = 0.27, Da = 0.73, H 0 = 70 km s -1 cosmology, as 
was used by Rusu et al. (2013). 

2.3.3. SDSS J1430+4105 

SDSS1430+4105 was first described by Bolton et al. 
(2008) as part of the SLACS survey. This system is 
at redshift zi ens = 0.285 with z source = 0.575, and has a 
complex morphology with several subcomponents as de¬ 
scribed by Eichner et al. (2012). Bolton reported an 
effective radius of 2.55” and a trsDSS=322 km s -1 . 

A very detailed lens model analysis of this system was 
then conducted by Eichner et al. (2012). This analy¬ 
sis was a direct, semi-independent comparative analysis 
using both Gravlens (LTM) and Lensview (non-LTM) 
software. The authors studied five different models us¬ 
ing Gravlens/Lensmodel, including an SIE and a Power 
Law (PL) model as well as three two-component de Vau- 
couleurs plus dark matter models. Similar results were 
found with the two different lens model analyses. They 
also studied four models using Lensview (Wayth and 
Webster 2006) including an SIE and PL models with and 
without external shear. We use the Gravlens/Lensmodel 
SIE model as the basis of the indirect comparison with 
their work. The plane of optimization used in the Eich¬ 
ner model is not explicitly stated in the report (Eichner 
et al. 2012). 

The models developed in the previous study were not 
available, and thus all models used were written for 
this study. The results referred to as Model I by Eich¬ 
ner did not use any priors in the lens model for SDSS 
J1430+4105, although priors were used in the develop¬ 
ment of the model with results within the error limits 
reported. Similarly, priors were not used in the mod¬ 
els in this study. The free parameters used by Eichner 
et al included the lensing strength b, the ellipticity and 
the orientation of the single-component SIE lens. These 
same free parameters were used in the models developed 
for this study. The positions of the multiple images of 
this system were taken from Table 2 in Eichner et al. 
( 2012 ) . 

This is both an indirect comparison (compared with 
the SIE model in the published study of (Eichner et al. 
2012 )) and direct comparisons of the four lens models 
studied here. All four models of this system used a D m = 
0.3, Da = 0.7, H 0 = 70 km s -1 cosmology, as was used 
by Eichner et al. (2012). 

2.3.4. J1000+0021 
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Using imaging data from CANDELS and the large 
binocular telescope, van der Wei and colleagues 
recently reported the quadruple galaxy-galaxy lens 
J100018.47+022138.74 (J1000+0221), which is the first 
strong galaxy lens at zi ens >1 (van der Wei et al. 
2013). This interesting system has a Zi ens = 1.53 and a 
%source — 3.417. 

van der Wei et al. (2013), analyzed this system in the 
manner described previously by van de Ven et al. (2010), 
and reported an Einstein radius of Re = 0.35” with an 
enclosed mass of Me = (7.6 ± 0.5) x 10 10 M 0 with an 
upper limit on the dark matter fraction of 60%. The 
highly magnified (40 x) source galaxy has a very small 
stellar mass (~ 10 8 M 0 ). The z = 1.53 lens is a flattened, 
quiescent galaxy with a stellar mass of ~ 6 x 10 10 M 0 . 

There have been no other lens model analyses of this 
system using software models and therefore all models 
were developed for this study using data from van der 
Wei et al. (2013), and is thus is a direct comparison of the 
four lens software models studied. There were no priors 
used in the lens models of J1000+0021 in this study. The 
free parameter in the SIS models was only the velocity 
dispersion. In the SIE models, free parameters included 
the velocity dispersion, orientation and ellipticity. The 
image positions in all models in this study for this system 
were taken from Table 2 in (van der Wei et al. 2013). 

All four models of this system used a Ll m = 0.3, LIa = 
0.7, H 0 = 70 km s -1 cosmology. 

2.4. Lens Models 

The analyses in this study were performed with four 
strong gravitational lens model software packages that 
have been used extensively in the literature. All four 
systems were modeled with all four lens model software 
packages. Lenstool and Lensmodel were executed under 
Scientific Linux version 6.4 (except as noted for Lens- 
model in section §3.6), and PixeLens and glafic were ex¬ 
ecuted under OS/X version 10.9. All of these lens model 
software codes were reviewed in the Orphan Lens Project 
and the descriptions of the software are from the web site 
(Moustakas and Brownstein 2013) as well as from a re¬ 
view of lens model software (Lefor et al. 2013). 

Error calculations were performed according to the 
method of Rusu et al (Rusu et al. 2013). The errors 
quoted for the calculated parameters (ellipticity, orien¬ 
tation, magnification, time delay, etc.) reflect the calcu¬ 
lations corresponding to calculations within the la con¬ 
fidence interval for velocity dispersions. 

The fit of the models is assessed by y 2 optimization 
and the RMS uncertainty. The RMS is calculated by: 

RM Simages E ((Xj Xi ) +( 2/3 Hi) ) / Nimagesj (1) 

i 

where x t and y i are the locations given by the model, 
and Xi and yi are the real images location, and the sum 
is over all N irnages images. The y 2 results are calculated 
for the models by Lenstool, Lensmodel and glafic, and 
are reported in the data tables. The RMS value is re¬ 
ported by Lenstool directly, while a manual calculation 
was necessary for models using Lensmodel and glafic. 

2.4.1. PixeLens 


PixeLens is a non-LTM strong gravitational lens model 
software that is available for download 4 as a Java pro¬ 
gram which runs in a browser window (Saha et al. 2006). 
Version 2.7 was used in these studies. PixeLens is accom¬ 
panied by a manual (Read 2012a) and a tutorial (Read 
2012b). PixeLens reconstructs a pixelated mass map for 
the lens in terms of the arrival time surface and has been 
used in several studies (Saha et al. 2006). 

PixeLens employs a built-in MCMC approach and cre¬ 
ates an ensemble of 100 lens models per given image con¬ 
figuration. The pixelated mass map offers the advantage 
of being linear in the unknown. Since all equations are 
linear in the unknowns, the best-fitting model and its un¬ 
certainties are obtained by averaging over the ensemble 
(Saha et al. 2006; Kohlinger and Schmidt 2013). The pix¬ 
elated mass map differentiates PixeLens from the other 
software used in this study which fit parametric func¬ 
tional forms. 

2.4.2. Lenstool 

Lenstool has been used in many different studies and 
is available for download 5 (Julio et al. 2007). Version 
6.7.1 was used in these studies. Lenstool has features of 
both LTM and non-LTM modeling and uses a Bayesian 
approach to strong lens modeling and has been well- 
described in the literature (Julio et al. 2007; Julio and 
Kneib 2009). There are several resources available for 
writing lens models for Lenstool (Kneib 2012; McCourt 
2006). 

Lenstool can optimize most of the parameters in a 
model. Models produced by HydraLens for Lenstool were 
modified slightly to add appropriate optimization param¬ 
eters and then used with Lenstool. Lenstool optimization 
is performed with MCMC. Lenstool uses the geometry 
of the images given and then finds counter-images. The 
image positions are recomputed and the time delays de¬ 
termined. 

2.4.3. Gravlens 

The Gravlens package includes two codes, Gravlens 
and Lensmodel (Keeton 2001b) accompanied by a user 
manual (Keeton 2004). Version 1.99o was used in these 
studies, under the Linux operating system, downloaded 
from the Astrophysics Source Code Library 6 . However, 
the Darwin (Macintosh) executable file provided for ver¬ 
sion 1.99o will only run on the now obsolete PowerPC ar¬ 
chitecture. A newer version to run on the Macintosh plat¬ 
form under OS/X 10.9 (Gravlens version dated Novem¬ 
ber 2012) was kindly provided by Professor Keeton, for 
these studies. Lensmodel is an extension of Gravlens and 
was used for all analyses here. It is fully described in two 
publications by Keeton (Keeton 2001b, a), and has been 
used extensively. 

Lensmodel is an LTM lens model software, which opti¬ 
mizes the selected lens parameters and uses a tiling algo¬ 
rithm and a simplex method with a polar grid centered 
on the main galaxy. The tiles are used to determine the 
image positions, and then uses a recursive sub-gridding 
algorithm to more accurately determine image positions. 

2.4.4. glafic 

Glafic is an LTM lens model software, and includes 
computation of lensed images for both point and ex- 
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tended sources, handling of multiple sources, a wide va¬ 
riety of lens potentials and a technique for mass model¬ 
ing (Oguri 2010) with multiple component mass models. 
Version 1.1.5 was used on the OS/X platform and version 
1.1.6 was used with Linux in these studies'. 

Each lens is defined by the lens model and seven pa¬ 
rameters. A large catalog of lens models is available (in¬ 
cluding point mass, Hernquist, NFW, Einsato, Sersic, 
etc.). After defining the parameters and the lens mod¬ 
els, parameters to be varied in the \ 2 minimizations are 
specified. Following this, the desired commands are is¬ 
sued such as computing various lensing properties, Ein¬ 
stein radius, write lensing properties to a FITS file, etc 
(Oguri 2013). Glafic has been used in a large number of 
lens model studies, including SDSSJ1004 (Oguri 2010), 
and perforins lens model optimization. 

Glafic uses a downhill simplex method of optimization. 
The image plane is divided using square grids by an adap¬ 
tive meshing algorithm. The level of adaptive meshing is 
set as an optional parameter. 

3. RESULTS 

Each of the four lens systems was modeled with all four 
lens model software codes including PixeLens, Lenstool, 
Lensmodel, and glafic. Best-fit lens model parameters 
from previous studies are presented along with the re¬ 
sults from this study for each system. The results re¬ 
ported for each lens were intended to follow the format 
of the data for best-fit lens parameters as reported in pre¬ 
vious studies, and therefore there are some differences in 
the data presented for the four lens systems. Lenstool 
and glafic directly calculate the velocity dispersion and 
then calculate the Einstein radius and mass within the 
Einstein radius. Lensmodel directly calculates the Ein¬ 
stein radius, from which the other values were deduced. 
PixeLens calculates mass at various distances from the 
lens mass. The figures shown are the output from each of 
the software packages used, and represent the graphical 
capabilities of that software. 

3.1. COSMOS J095930+023427 

Best-fit lens model parameters for COSMOS 
J095930+023427 are shown in Table 1. The data 
reported by Cao et al. (2013) are at the upper portion 
of the table, and show the results of the Lenstool model. 
The results in this study using the Lenstool model are 
somewhat different because the model in this study used 
optimization in the image plane, rather than the source 
plane optimization used by Cao. The glafic model was 
also conducted with optimization in the image plane, 
while the Lensmodel model is conducted with source 
plane optimization because image plane optimization 
did not yield a satisfactory model. Direct comparisons 
of the four software models evaluated are shown next. 
The models used here were based on the SIE+SIS+SIS 
model used by Cao et al. (2013). The Lenstool model 
includes an SIE potential at zi ens = 0.892, and two SIS 
potentials at zi ens = 0.7, as described by Cao et al. 
(2013). 

The PixeLens model used image coordinates from Cao 
et al. (2013), and calculated an enclosed mass inside 
the Einstein radius very close to that calculated by the 
Lenstool model. The Lenstool model optimized the ellip- 


ticity, position angle and velocity dispersion for the sin¬ 
gle SIE potential, and only the velocity dispersion for the 
two SIS potentials, as done by Cao et al. (2013) as free 
parameters. Lensmodel sets all three lens potentials at 
ziens= 0-892 because the software does not permit mul¬ 
tiple lens planes. The ellipticities and position angles 
optimized by each of the three codes are quite different. 

The Einstein radius of the SIE potentials are similar 
while there is some difference in the optimized velocity 
dispersions calculated by the three codes, particularly in 
the values calculated by glafic for the second potential. 
In an effort to understand this, the velocity dispersions 
of the first and second potentials were fixed at the values 
calculated by Lenstool at 234 and 412 km s -1 respec¬ 
tively and the velocity dispersion of the third potential 
allowed to optimize, using glafic. This resulted in a ve¬ 
locity dispersion of 632 km s -1 for the third potential. 
When the first and third values were fixed at 238 and 
603 km s -1 (as found by Cao), the second potential was 
optimized at 57 km s -1 . Magnifications and time delays 
for this model are shown in Table 2. Both time delays 
and magnifications calculated by all four models show 
great variability. 

The velocity dispersions shown in Table 1 as calculated 
here are slightly different from those reported by Cao 
et al. (2013), because of the different optimization tech¬ 
nique. The velocity dispersion values shown for the Lens¬ 
model and glafic models are somewhat different. The 
Lenstool model used by Cao (Cao et al. 2013) defined 
potentials at zi ens = 0.892 and 0.7, although Lenstool al¬ 
lows only a single lens plane (Kneib 2012). When the 
results were re-calculated defining all lenses in the same 
plane ( Zi ens = 0.892) using Lenstool, there was no ef¬ 
fect on the calculation of the velocity dispersion. The 
wide variation in time delays calculated for this system 
are shown in Table 2, and are consistent with the wide 
range in time delays reported in our previous study using 
different models (Lefor and Futamase 2013). There is a 
wide disparity in time delay calculations seen in all of 
the systems evaluated in this study. 

The image plane for a representative model calculated 
using Lenstool is shown in Figure 1. The image posi¬ 
tions change from the input positions because of the im¬ 
age tracing algorithm used. This slight difference may 
account for the differences seen in time delay and mag¬ 
nification. Lenstool identifies 16 total images, which are 
nearly superimposed at the original positions of the four 
images shown in Figure 1. 

Each of the models uses somewhat different optimiza¬ 
tion schemes, and the velocity dispersions are a result 
of optimization, which may explain some of the differ¬ 
ences shown in Table 1. The differences in the results 
among the three software programs is not surprising, 
since this model had all three velocity dispersions as free- 
parameters. 

3.2. SDSS J1320+16U 

Best-fit lens model parameters for SDSS J1320+1644 
are shown in Table 3 with an indirect/direct comparison 
to the study of Rusu et al. (2013) and the four direct 
comparisons in this study. Rusu et al. (2013) utilized a 
glafic model that modeled the potentials of Gl, G2 and 
G4 which were boosted by an embedding dark matter 
halo. One of the published models used four SIS poten- 
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TABLE 1 

Best-fit lens model parameters for COSMOS J095930+023427 


Software 

RA 

Dec 

x 2 

e 

e 

Re 

M(< R e ) 

<70 


(") 

(") 

RMS(") 


(deg) 

(") 

(10 11 Mq) 

(km s -1 ) 


Results from Cao et al. (2013) 

Lenstool 



1.7 






SIE 

[0.0] 

[0.0] 


0.28 

-10 

0.79 

3 49 +0,5 

238 

SIS 

[-10.98] 

[0.474] 






391 

SIS 

[3.52] 

[13.2] 






603 



Direct Comp, 

arison of Lens Models (This Study) 



PixeLens 







3.51 


Lenstool 



1.2 






SIE 

[0.0] 

[0.0] 

0.06 

0.33 ± 0.06 

-12 ±8 

0.79 ±0.03 

3.49 ±0.7 

234 ± 23 

SIS 

[-10.98] 

[0.474] 




1.8 

11.7 

412 ± 32 

SIS 

[3.52] 

[13.2] 




4.3 

67.9 

632 ± 48 

Lensmodel 



2.2 






SIE 

[0.0] 

[0.0] 

0.3 

0.002 ± 0.002 

84 ± 18 

0.76 ±0.06 

1.91 ± 0.6 

252 ± 23 

SIS 

[-10.98] 

[0.474] 




1.6 

8.81 

369 ± 36 

SIS 

[3.52] 

[13.2] 




2.3 

18.1 

442 ± 56 

glafic 



0.9 






SIE 

[0.0] 

[0.0] 

0.2 

0.50 ±0.08 

65 ± 2 

0.79 ±0.02 

2.06 ±0.8 

256 ± 23 

SIS 

[-10.98] 

[0.474] 




0.00 

* 

0.60 ±0.05 

SIS 

[3.52] 

[13.2] 




4.2 

57.6 

590 ± 48 


Note. — Values shown in square brackets are fixed in the models. Values without brackets are the 
optimized/calculated values from the model. ^Calculated mass at 6.10E±00 Mq 


TABLE 2 

Magnification and Time Delays for Four Images in 
COSMOS J095930+023427 


Software 

A 

B 

C 

D 

PixeLens 

Time Delay 

0 

0.7 

3.4 

0.07 

Lenstool 
Magnification 
Time Delay 

7.8 ±4.3 

0 

6.3 ± 10 
37 ± 22 

8.2 ±6.2 

31 ± 18 

7.9 ± 7.0 
32 ± 21 

Lensmodel 
Magnification 
Time Delay 

1.5 ± 1.5 

0 

1.3 ± 1.5 
16 ± 12 

0.7 ± 2.5 
9.4 ±8.8 

0.9 ± 1.5 
9.9 ± 7.5 

glafic 

Magnification 
Time Delay 

-4.2 ± 1.1 

0 

9.2 ± 2.2 
28 ± 7 

-102 ± 7.0 
5.0 ± 1.4 

113 ± 23 
4.9 ± 1.4 


Note. — Time delay is shown in days 


tials and fixed the locations of the first three, allowing 
the position of the fourth (the dark matter halo) to op¬ 
timize (”SIS free”). Furthermore, they concluded that 
any reasonable mass model reproduced the observed im¬ 
age configuration. The values shown in Table 3 are those 
as presented in the paper, as the ’SIS free’ model (Rusu 
et al. 2013). In this study, the values calculated by Rusu 
et al. (2013) and shown here were reproduced exactly 
using their model, and the ± values are at la. 

The PixeLens model has a much lower calculated time 
delay than the other models, and an enclosed mass within 


ler of the value reported by Rusu et al. (2013). As per¬ 
formed by Rusu et al. (2013), the positions of the sources 
were kept fixed for the first three SIS potentials. The ve¬ 
locity dispersion and position of the last potential (the 
dark matter halo) were optimized. The optimized po¬ 
sition of the fourth potential calculated in the Lenstool 
model is quite different, and the velocity dispersion is 
similar to other models. Lensmodel uses the Einstein 
radius, rather than velocity dispersion so the Einstein 
radii for the first three SIS potentials were fixed, and 
the fourth was a free parameter. The mass of the fourth 
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Fig. 1.— Image plane for COSMOS J095930+023427 calculated 
by Lenstool. The critical line is shown in red. Each image position 
is shown as a cross with a label. The center of the mass distribution 
is shown in gray at the center 

potential calculated by Lensmodel is nearly identical to 
the values calculated using glafic by Rusu et al. (2013) as 
well as the Lenstool and glafic models reported here. The 
time delays and magnification values show more variabil¬ 
ity. 

The Lenstool, glafic and Lensmodel models conducted 
in this study use image plane optimization, similar to the 
glafic analysis conducted by Rusu. The calculated mod¬ 
els of SDSS J1320+1644 show similar optimization for 
the mass of the fourth SIS potential, with fairly similar 
positions calculated by Lensmodel and glafic, while the 
positions calculated by Lenstool show greater variabil¬ 
ity. There is great variability among the calculated time 
delays and magnifications. 

The calculations performed in this study using glafic 
are the same as the glafic SIS-free model reported by 
Rusu et al. (2013). Table 3 shows that the mass calcu¬ 
lated for the fourth SIS potential, which was a free pa¬ 
rameter, optimized to the same value for Lenstool, Lens¬ 
model and glafic. The optimized geometry was slightly 
different for Lenstool compared to the others. The Ein¬ 
stein radius calculated by all four models was almost the 
same for the first SIS potential. The fact that the veloc¬ 
ity dispersion for the fourth lens potential was optimized 
to the same value in all of the models may reflect the fact 
that there was only a single free parameter in each model. 
This is different from the results above with COSMOS 
J095930+023427, which optimized three lens potentials 
as free parameters, with varying results among the mod¬ 
els tested. 

The model of SDSS ,11320+1644 was straightforward 
including four SIS potentials which was reproduced in all 
software models without difficulty. The model used by 
Rusu (Rusu et al. 2013) had 0 degrees of freedom and 
with a resulting y 2 << 1, due in part to the design of 
the model with 14 nominal constraints and 14 param¬ 
eters. The similarity of the potentials used to model 
the system may have contributed to the close results for 
optimization of the mass. Despite this, position, magni¬ 
fication and time delay showed great variability among 
the four models. The velocity dispersion for only the 
fourth lens potential was left as a free parameter, with 
the other three fixed, which is likely a major factor in 
the close agreement found among the various models in 
the calculation of the velocity dispersion. 

The image plane of the glafic model is shown in Figure 
2, which is the same as shown in Figure 6 of Rusu et al. 



Fig. 2.— Image plane for SDSS J1320+1644 calculated by glafic. 
The blue line is the critical line. Image positions are shown as red 
triangles. The centers of the masses are shown as black crosses. 

(2013). The image positions in the image plane are the 
same as the input positions in all models. Despite this, 
there is variability in the time delay and magnification 
calculations. 

3.3. SDSSJ1430+4105 

The indirect comparison to the work of Eichner 
et al. (2012) and the results of the four direct com¬ 
parisons in this study are shown in Table 4. In 
Eichner et al. (2012) there are five different models 
tested for SDSSJ1430+4105. The models were tested 
with Gravlens/Lensmodel (LTM) (Keeton 2001b) and 
Lensview (LTM) (Wayth and Webster 2006), and the 
results compared in a direct comparison. 

The model used in the current study is based on Model 
I, as described in Eichner et al. (2012), which models the 
lens as an SIE, ignoring the environment of the lens. The 
best fitting parameters reported by Eichner et al. (2012) 
are shown in Table 4. The results of Eichner are in good 
agreement with those by Bolton et al. (2008). In the 
SIE model using Lensview as reported by Eichner et al. 
(2012), their results were very similar to those with the 
Lensmodel model. The input files for the model used by 
Eichner et al. (2012) were not available for this study, 
making this study both an indirect and direct compari¬ 
son. 

The Lenstool, glafic and Lensmodel models conducted 
in this study use image plane optimization. The enclosed 
mass calculated by PixeLens inside the Einstein radius, 
is slightly higher than the result published by Eichner 
et al. (2012). The Einstein radii calculated by all the 
models are very close to each other as well as close to 
the result of Eichner et al. (2012). As shown in other 
lens systems in this study, there is considerable variation 
in magnification and time delay calculations among the 
four models studied as shown in Table 5. The optimized 
ellipticities among the four models are all quite close, 
but there is significant variability in the optimal position 
angles calculated. 

The models used in this study (results shown in Ta¬ 
bles 4 and 5) were written without detailed knowledge 
of the model used by Eichner et al. (2012). Despite this, 
the models all had similar results, especially in regard 
to Einstein radius, enclosed mass and velocity dispersion 
calculations. 

The image plane of the glafic model of this system is 
shown in Figure 3. The glafic (Figure 3) model resulted 
in just 4 images in the output image plane. In contrast, 
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TABLE 3 

Best-fit lens model parameters for SDSS J1320+1644 


Software 

RA 

Dec 

x 2 


At 

Re 

M{< Re) <?o 


(") 

(") 

RMS(") 


(days) 

(") 

(10 11 Mq) (km s" 1 ) 


Results from Rusu et al. (2013) 

glafic 



X 2 << 1 37 ±29 -860 ±460 

SIS 

-4.991 


0.117 

0.5 ±0.2 2.1 ±1.5 [237] 

SIS 

-2.960 


3.843 

[163] 

SIS 

-9.169 


5.173 


SIS 

-4.687 

1.149 

645 ± 25 


Direct Comparison of Lens Models (This Study) 


PixeLens [0.0] [0.0] 


Lenstool 

SIS 

SIS 

SIS 

SIS 


-4.991 

-2.960 

-9.169 

-0.471 


0.117 

3.843' 

5.173' 

0.179 


11.5 

2.3 


1.0 ± 1.2 


3.5 

-1 ± 10 


0.49 

0.23 

0.12 

3.6 


2.9 


1.1 

0.25 

0.07 

61 


237 

163 

118 

645 ± 25 


Lensmodel 
SIS [-4.991] 

SIS [-2.960] 

SIS [-9.169] 

SIS -3.93 


[0.117] 

[3.843] 

[5.173] 

2.43 


51.1 

3.9 


1.3 ±2.0 485 ±210 


[0.49] 

[0.23] 

[ 0 . 12 ] 

2.9 


1.1 [237] 

0.25 [163] 

0.07 [118] 

53 576 ± 25 


glafic 


SIS 

[-4.991] 

SIS 

[-2.960] 

SIS 

[-9.169] 

SIS 

-4.687 


2.0E-06 37 ±29 

117] 0.10 

843] 

173] 

149 


-860 ± 460 

[0.5] 
[0.23] 
[ 0 . 12 ] 
3.6 ±0.2 


2.1 ± 1.5 [237] 

0.25 [163] 

0.070 [118] 

61 645 ± 25 


Note. — Values shown in square brackets are fixed in the models. Values without brackets are the 
optimized/calculated values from the model. 



Fig. 3.— Image plane for SDSS J1430±4105 calculated by glafic. 
The blue line is the critical line. Image positions are shown as red 
triangles. The center of the mass density is shown as a black cross. 

Lenstool identified a total of 28 images. 

The position angles were somewhat different but there 
was good agreement among the models for ellipticity cal¬ 
culations. As with other models in this study, there was 
variation in the calculation of time delays and magnifi¬ 
cations. 

One of the reasons for such close agreement among the 
models is that the models all used a single SIE potential, 
which allowed for comparable potentials among the four 
lens model codes tested. There was a single lens plane 
in all of the models. 

3.4. J1000+0021 


An analysis of this lens system was performed by van 
der Wei et al. (2013) with a calculated Einstein radius of 
Re = 0.35” (or 3.0 kpc) with an enclosed mass of M E = 
7.6 ± 0.5 x 10 10 Mq. There have been no extensive lens 
model analyses of this system published to date. This is 
the first strong galaxy lens at Zi ens >1. In all models, the 
position (both RA and Dec) of the lens galaxy was kept 
constant, and the mass was a free parameter optimized 
by the software. Further details of the model used were 
not provided, such as the model software used or the y 2 
calculation. 

Results of the four direct comparisons done in this 
study are shown in Table 6. This lens system was mod¬ 
eled both using an SIS and an SIE, with all lens model 
software tested. The Lenstool, glafic and Lensmodel 
models conducted in this study use image plane optimiza¬ 
tion. The PixeLens model calculated the enclosed mass 
the same as reported by van der Wei et al. (2013). Using 
an SIS potential, the Einstein radius, enclosed mass and 
velocity dispersion calculations were nearly the same for 
Lenstool, Lensmodel and glafic. The Einstein radii and 
velocity dispersions were very close to that reported by 
van der Wei et al. (2013). Calculations of magnification 
and time delay showed quite a bit of variability in these 
models. 

The results of the models shown in Table 6 show very 
similar results for the SIS and the SIE models. The 
enclosed mass within the Einstein radius is somewhat 
lower than that reported by van der Wei et al. (2013) for 
Lenstool, Lensmodel and glafic although the PixeLens 
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TABLE 4 

Best-fit lens model parameters for SDSSJ1430+4105 


Software 

RA 

Dec 

x 2 

e 

e 

Re 

M(< R e ) 

^0 


(") 

(") 

RMS(") 


(degrees) 

(") 

( 10 11 M 0 ) 

(km s -1 ) 


Results from Eichner et al. (2012) 

Lensmodel 

SIE 

[ 0 . 0 ] 

[ 0 . 0 ] 

11.5 

0 7V + 0 - 02 
u -' - 0.02 

- 21 - 6 la.S 

1 49+ 002 
±, ^ y - 0.02 

c ok + 0.07 
0.06 

322 ± 22 

Direct Comparison of Lens Models (This Study) 

PixeLens 

[ 0 . 0 ] 

[ 0 . 0 ] 





6.04 


Lenstool 



4.9 






SIE 

[ 0 . 0 ] 

[ 0 . 0 ] 

0.25 

0.89 ± 0.03 

82 ± 22 

1.45 ±0.02 

3.59 ±0.05 

317 ± 22 

Lensmodel 



15.9 






SIE 

[ 0 . 0 ] 

[ 0 . 0 ] 

0.30 

0.53 ±0.32 

22 ± 32 

1.42 ±0.02 

3.51 ± 0.05 

309 ± 22 

glafic 



2.4 






SIE 

[ 0 . 0 ] 

[ 0 . 0 ] 

0.29 

0.52 ± 0.03 

-10 ±3.4 

1.50 ± 0.02 

3.67 ± 0.11 

334 ± 22 


Note. — Values shown in square brackets are fixed in the models 


TABLE 5 

Magnification and Time Delays for Five Images in SDSS J1430+4105 


Software 

A 

B 

C 

D 

E 

PixeLens 

Time Delay 

0 

0 

0 

0 

0 

Lenstool 
Magnification 
Time Delay 

1.10 ± 0.2 

0 

2.66 ± 0.1 

82 ± 18 

0.5 ±0.5 

66 ±22 

2.7 ± 0.4 

90 ±33 

0.49 ± 0.3 
-16 ± 12 

Lensmodel 
Magnification 
Time Delay 

1.0 ± 0.0 

0 

1.56 ±0.2 

31 ± 2.5 

0.7 ± 0.15 

32 ±3.0 

0.77 ± 0.15 
32 ± 3.0 

0.36 ±0.10 
39 ± 3.5 

glafic 

Magnification 
Time Delay 

2.8 ± 0.02 

0 

-1.4 ±0.02 
34 ± 4 

-1.4 ±0.02 
34 ±4 

1.1 ± 0.02 

0 



Note. — Time delay is shown in days 


model reproduced the enclosed mass calculation very 
well. Similar to the models used for SDSSJ1430+4105, 
these models were all quite straightforward with a sin¬ 
gle potential located at the origin, which may have con¬ 
tributed to the concordance of results. 

Comparing the results of the SIE models, the results 
with an SIE model using the four software packages were 
also nearly identical, although among the SIE models, 
there was some variability in the calculations of ellipticity 
and position angle. 

The image plane of the glafic model of this system is 
shown in Figure 4. This system is particularly interesting 
as the image positions in the Lensmodel and glafic mod¬ 
els have an almost identical geometry, while the image 
positions in the Lenstool model are different. The time 
delays and magnifications in the Lensmodel and glafic 
models are very similar, while the Lenstool model values 
are different. 



Fig. 4.— Image plane for J1000+0021 calculated by glafic. The 
blue line is the critical line. Image positions are shown as red 
triangles. The center of the mass density is shown as a black cross. 

3.5. Comparison of Results 

There are some generalizations that can be made com¬ 
paring the results calculated from the models for each 
of the four lens systems studied. The Einstein radii and 
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TABLE 6 

Best-fit lens model parameters for J1000+0021 


Software RA Dec 

x 2 

P 

At 

Re 

M(< Re) ctq 

(") (") 

RMS(") 


(days) 

(") 

(10 11 M 0 ) (kms- 1 ) 



Results from van der Wei et al. (2013) 






40 ± 2 


0.35 

0.76 ± 0.5 

182 ± 10 

Direct Comparison of Lens Models (This Study) 

- Singular Isothermal Sphere (SIS) Models 

PixeLens 

[ 0 . 0 ] 

[ 0 . 0 ] 



2.3 


0.8 


Lenstool 

SIS 

[ 0 . 0 ] 

[ 0 . 0 ] 

2.9 

1.6 ±3.3 

-18 ±22 

0.4 ±0.2 

0.6 ± 0.2 

200 ± 10 


0.05 






Lensmodel 

SIS 

[ 0 . 0 ] 

[ 0 . 0 ] 

5.8 

2.3 ±0.4 

0.1 ± 0.1 

0.4 ± 0.4 

0.6 ±0.4 

190 ± 10 


0.23 






glafic 

SIS 

[ 0 . 0 ] 

[ 0 . 0 ] 

0.3 

7.1 ± 12 

3.8 ± 1.8 

0.4 ± 0.05 

0.6 ±0.25 

192 ± 10 


0.10 






Singular Isothermal Ellipsoid (SIE) Models 

Software 

RA 

Dec 

x 2 

e 

e 

Re 

M(< R e ) 

0-0 


(") 

(") 

RMS(") 



(") 

(10 11 M 0 ) 

(km s -1 ) 

Lenstool 

SIE 

[ 0 . 0 ] 

[ 0 . 0 ] 

1.7 

0.76 ±0.4 

19 ± 18 

0.4 ± 0.05 

0.6 ±0.07 

190 ± 10 


0.04 






Lensmodel 

SIE 

[ 0 . 0 ] 

[ 0 . 0 ] 

1.7 

0.26 ±0.04 

-71 ± 35 

0.4 ± 0.1 

0.6 ± 0.05 

190 ± 10 


0.12 






glafic 

SIE 

[ 0 . 0 ] 

[ 0 . 0 ] 

0.5 

0.008 ± 0.003 

-70 ± 19 

0.4 ± 0.05 

0.6 ± 0.22 

189 ± 10 


0.05 







Note. — Values shown in square brackets are fixed in the models 


mass within the Einstein radii are quite close for the four 
models of each system. The Einstein radius is calculated 
from the average distance between the lens center and 
multiple images, and is insensitive to the radial density 
profile (Oguri et al. 2013). The conversion from the Ein¬ 
stein radius to the enclosed mass within the Einstein ra¬ 
dius is dependent only on the lens and source redshifts, 
and is therefore model independent (Oguri et al. 2013). 
Thus, the similar results for Einstein radii and mass 
within the Einstein radii are expected since all models 
had the same system geometry of zi ens and z source . 

There is variation among the calculated time delays 
and magnifications comparing the models generated by 
each of the four lens model software programs. The im¬ 
age positions input to each model were identical. The 
image positions in the models studied change due to the 
ray-tracing algorithms in each software model. These 
differences explain some of the variation seen in time de¬ 
lay and magnification. In some cases, the use of a simi¬ 
larly parameterized model leads to a model that has not 
converged appropriately, which illustrates some of the 
differences in the software. This is evident in the RMS 
values calculated for the Lenstool and Lensmodel models 
of J1320+1644 

There is also little agreement among calculations of 
ellipticity and position angle. The variation in results 
for calculated ellipticity and position angle may be a re¬ 
sult of differences in the optimization algorithms used by 
Lenstool, Lensmodel and glafic. 


The complexity of the model also has an impact 
on agreement among the calculated values for veloc¬ 
ity dispersion. In the models for SDSS J1430+4105, 
J1000+0021and SDSS J1320+1644, there was only one 
potential with the velocity dispersion as a free-parameter 
for optimization. In all three of these systems, there 
was close agreement among the calculated values. In the 
model of COSMOS J095930+023427, there were three 
lens potentials which were optimized, with quite a bit 
of variation among the results from the three software 
programs used. 

3.6. Comparison of Lens Model Software by Version 

In order to evaluate the effect of software version 
and/or operating system / hardware platform, the model 
of SDSS J1320+1644 was evaluated with glafic and Lens¬ 
model on two different hardware platforms. Glahc is dis¬ 
tributed as an executable file with version 1.1.5 for the 
OS/X platform and version 1.1.6 for Linux. Lensmodel 
is available as an executable file only for download as ver¬ 
sion 1.99o for the Linux platform, and we were provided 
a version to run on OS/X. 

Input files for the models of SDSS J1320+1644 were 
used unchanged. In the first test, the model was tested 
with the two versions of glafic. The mass of the first 
three SIS potentials were held as fixed parameters and 
the mass of the fourth potential, as well as its position, 
were free parameters to be optimized. Identical results 
were reported using either version of glafic, on both plat- 
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forms. The results were identical including the numbers 
of models used for optimization in each run and the cal¬ 
culation of all parameters evaluated. The content of all 
output files produced by both versions was identical. The 
models for SDSS J1320+1644 were then tested with each 
of the two versions of Lensmodel. In this same test, opti¬ 
mizing the fourth SIS potential, results with Lensmodel 
were slightly different comparing the two versions. The 
optimized Einstein radius of the fourth potential using 
the Linux version is reported as 3.622605, and the OS/X 
version reports 3.622528. There are similarly small dif¬ 
ferences in the optimized position of the fourth potential. 

In the next test, the mass of all four potentials was 
optimized. The results with glafic, on both hardware 
platforms, were again identical in regard to all parame¬ 
ters evaluated, to the accuracy of the last decimal place 
reported. The contents of all output files produced by 
glafic were identical with the Linux and OS/X versions. 
However, the two versions of Lensmodel reported widely 
disparate results with the two versions tested. The Ein¬ 
stein radii of the four optimized SIS potentials using the 
Linux version are 1.851, 1.004, 0.3161 and 1.660. Using 
the OS/X version, the four potentials are optimized at 
2.234, 1.818, 0.3139 and 2.006. 

Among the various studies reported in Tables 7 and 8, 
the software version used is reported in only one study. 
The hardware platform and/or operating system used 
in the calculations is not reported in any of the studies 
shown in these tables. 

4. DISCUSSION 

Small changes in redshift have different effects on the 
calculation of time delays and mass by different lens 
model software codes (Lefor and Futamase 2013). In 
that study, a mock model with a single potential and 
four images as well as a model of SDSS J1004+4112 were 
evaluated and the effect of changes in redshift on changes 
in calculations of time delay and mass were determined. 
The study showed that changes in time delay and mass 
calculations are not always proportional to changes in 
DdAj/Dds, as would be predicted. The image positions 
change expectedly as a result of ray-tracing tracing al¬ 
gorithms which are not the same for all of the software 
used. This is partly responsible for the differences in the 
values of time delay and mass in both systems when com¬ 
paring the models from four different lens model software 
packages. 

The present study was designed to specifically compare 
the results using the same models with different software, 
rather than changes in the results, to compare results 
from different codes. The present study is the largest 
strong gravitational lens software comparison study per¬ 
formed to date, evaluating four different lens systems 
with four different lens model software codes in a sin¬ 
gle study, and is the first study to use HydraLens for the 
preparation of multiple models. 

4.1. Indirect Comparison Studies 

Table 7 shows a review of the existing literature where 
parameters have been calculated using strong gravi¬ 
tational lens models and compared with other pub¬ 
lished results, and as such are referred to as ’’indi¬ 
rect comparison studies”. In the indirect comparison of 
COSMOSJ095930 performed by Cao et al. (2013) and 


Faure et al. (2011), both analyses were conducted with 
Lenstool, and had very similar results for Einstein radius, 
mass enclosed within the Einstein radius, and other pa¬ 
rameters. It is difficult to discern the details of the model 
used by Faure et al. (2011) with regard to number, type 
and geometry of the lens potentials used. Indirect com¬ 
parisons are further complicated by a lack of available 
detail of the model used, making it difficult to reproduce 
previous results. 

4.2. Direct Comparison Studies 

Table 8 shows previous studies where different lens 
models were compared in the same study, as well as the 
evaluations performed in the present study, all of which 
constitute ’’direct comparison studies”. The direct com¬ 
parisons performed of Abell 1703, MS1358, MACSJ1206 
and SDSS120602 have been described in detail in Lefor 
and Futamase (2013) . The information in these di¬ 
rect studies was complementary in nature, leading to 
a greater understanding of the lens system. The lens 
SDSSJ1430 was investigated by Eichner et al. (2012) 
who compared the results using Lensview and Lens¬ 
model. The Lensmodel analysis assumes point sources 
while Lensview uses the two-dimensional surface bright¬ 
ness distribution of the same system. Both analyses led 
to the same conclusions regarding the mass distribution 
of the galaxy. The two lens model techniques were indeed 
complementary and led to similar results. In a compar¬ 
ative analysis of RX J1347.5-1145 using glafic and Pix- 
eLens, the authors note a 13 percent difference in the 
calculation of mass enclosed within the Einstein radius 
(Kohlinger and Schmidt 2013). They suggest that the 
LTM model used by glafic may not be assigning suffi¬ 
cient mass to the profiles in the models used. We ob¬ 
served a similar underestimation of enclosed mass by 
non-LTM models as compared to PixeLens in the analy¬ 
sis of J1000+0021. 

5. CONCLUSIONS 

Indirect comparison studies are of value, but as some 
of the comparisons conducted in this study show, it may 
be difficult to reproduce the results of previous studies 
without previous model files available to create the mod¬ 
els for other software, thus limiting the nature of the 
comparisons performed. In the analyses of COSMOS 
J095930+023427 and SDSS J1320+1644, being able to 
use the same models as used in the original studies, qual¬ 
ifies these as direct comparisons. This supports the im¬ 
portance of sharing lens model files in future studies. 

Even in direct comparisons, the results with one model 
may not be exactly the same as with another because of 
the difficulty in translating some of the features of one 
model to another because of the differences in features 
of the available software. For example, it is not possi¬ 
ble to parameterize a PixeLens model exactly the same 
as a Lenstool model because of inherent differences in 
the software. These differences may explain the observa¬ 
tions of (Kohlinger and Schmidt 2013) as well as some 
of the results in this study. Despite best efforts to sim¬ 
ilarly parameterize two models, there still may be small 
differences. This suggests that using several models to 
understand a system may lead to improved understand¬ 
ing. 
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TABLE 7 

Indirect Comparison Studies of Strong Gravitational Lens Models 


Lens System 

Software/Reference 

Publications 

Software/Reference 

Software/Reference 

Abelll689** 

LensPerfect 

Coe et al. (2010) 

ZB 

Broadhurst et al. (2005) 

PixeLens 

Saha et al. (2006) 

SDSS J1004** 

glafic 

GRADE 

PixeLens 


Oguri (2010) 

Liesenborgs et al. (2009) 

Saha et al. (2006) 

COSMOS J095930 

* 

Lenstool 

Lenstool 


* 

Cao et al. (2013) 

Faure et al. (2011) 

SDSSJ1430 

* 

Lensmodel / Lensview** 



* 

Eichner et al. (2012) 


SDSSJ1320 

* 

glafic 



* 

Rusu et al. (2013) 



* indicates present study with PixeLens, Lenstool, Lensmodel, and glafic, ** indicates that 
other studies of this lens have been published but are not listed. **Lensview is described 
in Wayth and Webster (2006) 


TABLE 8 

Direct Comparison Studies of Strong Gravitational Lens Models 


Lens System 

Reference 

i 

Software 

2 3 4 5 


SDSSJ1430 

AbelU703 

MS1358 

MACSJ1206 

SDSS120602 

RXJ1347.5 

Eichner et al. (2012) 

Zitrin et al. (2010) 

Zitrin et al. (2011) 
Umetsu et al. (2012) 

Lin et al. (2009) 
Kohlinger and Schmidt (2013) 

Lensview** 

ZB 

ZB 

ZB 

Lensmodel 

glafic 

Lensmodel 

GRALE 

GRALE 

Lenstool LensPerfect PixeLens SaWLens*** 

Lensview** 

PixeLens 


J1000+0221 

* 

PixeLens 

Lenstool 

glafic 

Lensmodel 

SDSSJ1430 

* 

PixeLens 

Lenstool 

glafic 

Lensmodel 

SDSSJ1320 

* 

PixeLens 

Lenstool 

glafic 

Lensmodel 

COSMOS J095930 

* 

PixeLens 

Lenstool 

glafic 

Lensmodel 


* indicates the present study. **Lensview is described in Wayth and Webster (2006) 


In seeking agreement among various models, the num¬ 
ber of free parameters for the lens potentials is an 
important factor. While there was reasonable agree¬ 
ment among the calculated values for Einstein radius 
in single potential models, such as SDSS J1430+4105 
and J1000+0021 in this study, there was less agree¬ 
ment in a more complicated model such as COSMOS 
J095930+023427, which may be a reflection of using more 
lens potentials to describe the system. 

Differences noted in time delay and magnification cal¬ 
culations may be due to differences in the image tracing 
algorithms used by each of the software models. The 
input image positions are the same in all models. The 
software calculates new positions based on the software 
specific ray-tracing algorithm used going from the source 
plane back to the image plane, resulting in differences in 
time delay results. The differences in optimization algo¬ 
rithms used also leads to some of the observed differences 
among the software models, with great variation in the 
calculation of ellipticity and position angle. 

These results demonstrate that there are significant 
differences in results using lens models prepared with dif¬ 
ferent software, and are consistent with a previous study 
of differences in lens models (Lefor and Futamase 2013). 
There is no intention to suggest that a particular group of 


models are necessarily more correct, but only to suggest 
that future lensing studies should evaluate lens models 
using several approaches to understand the system more 
thoroughly, as already being conducted in the Hubble 
Frontier Fields project. 

Based on the results of this study, in order to al¬ 
low comparisons across studies, it will be important to 
use a consistent nomenclature for lensing studies, spec¬ 
ifying indirect vs. direct comparisons, independent vs. 
semi-independent comparisons and the type of model 
being used as LTM vs. non-LTM, as we have previ¬ 
ously described (Lefor and Futamase 2013). Further¬ 
more, this study has shown at least in one situation that 
the software version used can significantly affect the re¬ 
sults which stresses the importance of specifying the soft¬ 
ware version number being used in all future studies, in 
addition to the hardware/operating system platform. It 
is also suggested that more detail is provided in future 
studies to allow reproducibility of the models such as the 
number and types of potentials used along with the name 
of the potential used in the various software packages. 

One of the most important aspects of any scientific ex¬ 
periment is reproducibility. In gravitational lens model 
studies, this is impossible in many cases because the soft¬ 
ware is not available to other investigators, or the lens 
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model files are not available. Code-sharing of software in 
astrophysics is essential, as emphasized by Shamir et al. 
(2013). Based on the studies reported here, the shar¬ 
ing of lens model files in gravitational lens studies is also 
essential to assure reproducibility and increased trans¬ 
parency in future gravitational lensing studies. Another 
approach in lensing studies that has been successfully 
applied in weak lensing is computer challenges. The use 
of multiple approaches including comparative studies of 
lens models, open software, open lens model files, and 
computer challenges will help to assure increased trans¬ 
parency in future studies and enhance the results. 
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